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Abstract 

Background: In a recent individual patient data meta-analysis, acupuncture was found to be superior to both sham and 
non-sham controls in patients with chronic pain. In this paper we identify variations in types of sham and non-sham controls 
used and analyze their impact on the effect size of acupuncture. 

Methods: Based on literature searches of acupuncture trials involving patients with headache and migraine, osteoarthritis, 
and back, neck and shoulder pain, 29 trials met inclusion criteria, 20 involving sham controls (n = 5,230) and 18 non-sham 
controls (n = 14,597). For sham controls, we analysed non-needle sham, penetrating sham needles and non-penetrating 
sham needles. For non-sham controls, we analysed non-specified routine care and protocol-guided care. Using meta- 
regression we explored impact of choice of control on effect of acupuncture. 

Findings: /Kcupuncture was significantly superior to all categories of control group. For trials that used penetrating needles 
for sham control, acupuncture had smaller effect sizes than for trials with non-penetrating sham or sham control without 
needles. The difference in effect size was -0.45 (95% C.I. -0.78, -0.12; p = 0.007), or -0.19 (95% C.I. -0.39, 0.01; p = 0.058) 
after exclusion of outlying studies showing very large effects of acupuncture. In trials with non-sham controls, larger effect 
sizes associated with acupuncture vs. non-specified routine care than vs. protocol-guided care. Although the difference in 
effect size was large (0.26), it was not significant with a wide confidence interval (95% C.I. —0.05, 0.57, p = 0.1). 

Conclusion: Acupuncture is significantly superior to control irrespective of the subtype of control. While the choice of 
control should be driven by the study question, our findings can help inform study design in acupuncture, particularly with 
respect to sample size. Penetrating needles appear to have important physiologic activity. We recommend that this type of 
sham be avoided. 



Citation: MacPherson H, Vertosick E, Lewith G, Linde K, Sherman KJ, et al. {2014} Influence of Control Group on Effect Size in Trials of Acupuncture for Chronic 
Pain: A Secondary Analysis of an Individual Patient Data Meta-Analysis. PLoS ONE 9(4): e93739. doi:10.1371/journal.pone.0093739 

Editor: Yu-Kang Tu, National Taiwan University, Taiwan 

Received November 13, 2013; Accepted March 6, 2014; Publislied April 4, 2014 

Copyriglit: © 2014 MacPherson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits 
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. 

Funding: The Acupuncture Trialists' Collaboration is funded by an R21 (AT004189I from the National Center for Complementary and Alternative Medicine 
(NCCAM) at the National Institutes of Health (NIH) to Dr Vickers) and by a grant from the Samueli Institute. Dr MacPherson's work on this project was funded in 
part by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme (RP-PG-0707-10186). Dr. Witt's work has been 
supported by the Carstens Foundation within the grant for the Chair for Complementary Medicine Research. The views expressed in this publication are those of 
the author{s) and not necessarily those of the NCCAM, NHS, NIHR or the Department of Health in England. The funders had no role in study design, data collection 
and analysis, decision to publish, or preparation of the manuscript. 

Competing interests: The authors have declared that no competing interests exist. 

* E-mail: hugh.macpherson@york.ac.uk 

11 Membership of the Acupuncture Trialists' Collaboration is provided in the Acknowledgments. 



Introduction 

One of the challenges of conducting a non-pharmacological 
clinical trial is choosing an appropriate control intervention. The 
simplest control arm is to offer patients routine clinical care 
without the experimental treatment. This controls for the expected 
course of the disease. However, the control arm can also be 
designed to control for other factors, for example, the non-specific 



effects associated with the time and attention that a patient 
receives from a clinician. 

The choice of control is particularly problematic in acupunc- 
ture, which has seen a large increase in published trials in recent 
years [1]. In an individual patient data meta-analysis of high 
quality trials conducted by the Acupuncture Triahsts' Collabora- 
tion [2], acupuncture reduced pain scores by 0.15 to 0.23 standard 
deviations in comparison to sham (placebo) acupuncture. When 
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the control group did not involve sham, efiect sizes ranged from 
0.42 to 0.57 [2]. Yet within the general categories of control - 
those with sham and those without sham - there were marked 
differences in the exact nature of the intervention received in the 
control group. For example, trials with sham control included 
those with acupuncture needles inserted at points not thought to 
be active, needles that did not penetrate the skin, and non-needle 
approaches, such as detuned electrical devices. For trials without 
sham control, the control group in some were simply advised to 
"avoid acupuncture"; in other trials, both acupuncture and 
control groups were offered additional treatment, such as physical 
therapy for back pain. 

In this paper, we aim to conduct an analysis of the Acupuncture 
Trialists' Collaboration dataset to determine how trial results vary 
by type of control. Specifically, we sought to determine the extent 
that effect sizes varied depending on whether needles were used for 
sham acupuncture, whether they penetrated the skin, and whether 
they were placed at or away from true acupuncture points. We 
also sought to determine whether there was variation in the effects 
of acupuncture associated with controls that did not involve sham, 
comparing "routine care", such as rescue medication made 
available to patients in both arms of the trial, with "protocoUed 
care" where the control treatment was a standard care specified in 
the study protocol. Establishing effect sizes associated with 
commonly used types of controls will be of value in informing 
future clinical trial design for acupuncture, as well as helping the 
interpretation of published trial results. 

Methods 

Included Trials 

Trials included in these analyses were identified through a 
systematic literature review that has been previously described [2] . 
The initial search was to November 2008, followed by a 
subsequent one conducted in December 2010. The searches 
included trials of acupuncture in four specified chronic pain 
conditions - non-specific musculoskeletal pain, shoulder pain, 
osteoarthritis, and chronic headache - where allocation conceal- 
ment was determined unambiguously to be adequate. For trials of 
musculoskeletal pain, it was additionally specified that the current 
episode of pain must be of at least four weeks' duration. The 
search resulted in the identification of 3 1 trials. 

Data Acquisition 

Individual patient data were obtained from 29 trials. Data on 
the trial-level characteristics of the controls were obtained direcdy 
from trialists. Twenty trials with 5,230 patients had controls in the 
form of a sham acupuncture arm (Table 1), and 18 trials with 
14,597 patients had non-sham controls (Table 2). 

Outcome 

The primary outcome used for this analysis was the primary 
pain endpoint as defined by the study authors. Where multiple 
criteria were considered in the primary outcome or if the primary 
outcome was inherendy categorical, we used a continuous measure 
of pain intensity measured at the same time point as the original 
primary' outcome. To make the various outcome measurements 
comparable between difierent trials, the primary endpoint of each 
was standardized by dividing by pooled standard deviation. 

Types of Sham Acupuncture Controls 

The characteristics we aimed to study in those trials with a sham 
acupuncture control group included whether or not a needle was 
used, whether a needle that penetrated the skin was used, whether 



sham was performed on true acupuncture points or non- 
acupuncture points, and whether needle insertion was deep or 
superficial. Information on acupuncture characteristics was 
obtained from the trial manuscript supplemented by a question- 
naire sent to trialists. 

Trials were classified as "needle sham" if it was reported that 
either a penetrating or non-penetrating needle was used for sham 
acupuncture. A non-penetrating needle is a device specially 
developed for acupuncture research in which the needle retracts 
into the handle rather than penetrating the skin; however, the 
pressure of the needle against the skin is a very similar sensation to 
insertion. "Non-needle sham" included trials using non-needle 
methods of sham acupuncture, such as an inactivated laser or 
transcutaneous electric nerve stimulation (TENS) device. Needle 
sham trials were further classified as to whether or not the needle 
used in the sham acupuncture group penetrated the skin. 
Penetrating needles were almost always inserted at locations away 
from true acupuncture points (thereby investigating point location) 
while non-penetrating sham needles were either applied at the 
same points as in the true acupuncture group (testing exclusively 
skin penetration and not location) or at non-acupuncture points 
(investigating penetration and location simultaneously). For 
example, in the trial of Linde et al [3], needles were inserted 
superficially away from true acupuncture points; in contrast, the 
sham technique in the Kleinhenz et al [4] trial consisted of a 
special needle that retracted into the handle rather than 
penetrating through the skin at true acupuncture points. 

We initially planned to investigate two other features of sham 
control: whether the depth of insertion for penetration was 
categorized by trialists as superficial or deep and whether sham 
was applied at or away from true acupuncture points. However, as 
shown in Table 1, only one trial reported using deep insertion in 
sham acupuncture [5]. For point location, there was strong 
coUinearity with sham technique, with only techniques avoiding 
skin penetration using true acupuncture points. 

As a sensitivity analysis, we re-analyzed the data excluding four 
trials which were determined by consensus among external 
reviewers as having an "intermediate likelihood of unblinding" 
[6] [7] [8] [9]. However, after excluding these trials, only one 
remaining trial used non-needle sham acupuncture, limiting our 
ability to use meta-regression. 

Types of Non-sham Controls 

Trials that included controls without sham were categorized 
into two types: "routine care" and "protocoUed care." Trials were 
identified as "routine care" if patients in both treatment and 
control groups had access to non-specified care as needed, such as 
rescue medications or other conventional care, but the use of such 
treatment was at the discretion of patients and doctors, with no 
specification in the protocol as to what treatments patients could 
receive. If protocols proscribed some treatments, such as surgery, 
but did not make specific recommendations as to allowable 
treatments, trials were defined as "routine care". Control groups 
where treatment consisted of information or education given to a 
patient ("attention control") were also considered to be routine 
care control groups. Trials were considered to be "protocoUed 
care" if the care in the control group was specified in the study 
protocol. This was typicaUy when the acupuncture group and the 
usual care control group both received an additional non- 
acupuncture treatment that was specificaUy indicated as part of 
the trial protocol. For example, trials that studied the effect of 
acupuncture and physical therapy compared to physical therapy 
alone were categorized as protocoUed care. 
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Table 1. Sham Acupuncture-Controlled Trials, by Types of Sham Control. 





Needle Used? 


Penetrating? 


True Acupuncture Points? 


DeptKi of insertion? 


Triais 


Yes 


Yes 


No 


Superficial 


Linde (2005) [3], Melchart (2005) [25], Diener (2006) [6], 
bchart (2006) L2dJ, Haake (200/) i\ IJ, bndres (200/) L2/J, Witt 
(2005) [28], Brinkhaus (2006) [29] 


Yes 


Yes 


No 


Deep 


Berman (2004) [5] * 


Yes 


No 


No 


N/A 


Vas (2008) [14] 


Yes 


No 


Yes 


N/A 


Foster (2007) [24], Guerra (2004) [30], Kennedy (2008) [31], 
Kleinhienz (1999) [4], Vas (2004) [12], Vas (2006) [13] 


No 


No 


No 


N/A 


Carlsson (2001) [9], Kerr (2003) [8] 


No 


No 


Yes 


N/A 


Irnich (2001) [7], White (2004) [32] 



*ln this trial, penetrating needles were used on non-acupuncture points and non-penetrating needles were used on true acupuncture points. For the main analysis, we 
considered this trial as penetrating-needle sham used on non-acupuncture points. 
doi:l 0.1 371 /journal.pone.0093739.t001 



In two trials there was a disagreement about whether the close 
specification of mecUcation and other treatment in the control 
group constituted "protocoUed care" or an active control group, 
which are excluded from analyses as per the review protocol [10]. 
The trial of acupuncture for migraine by Diener et al. [6] had a 
control group in which patients received standard pharmacolog- 
ical therapy for migraine prophylaxis. However, acupuncture and 
sham acupuncture groups did not receive prophylactic medication. 
The trial of acupuncture for lower back pain by Haake et al. [1 1] 
offered "routine management" up to and including physiotherapy, 
drugs and exercise to control group patients. Acupuncture and 
sham acupuncture patients had the same access to rescue 
medication as the "routine management" group, but did not 
have access to the same physiotherapy sessions or exercise 
consultations with physicians. 

Statistical Methods 

We used random-effects meta-regression to test the effect of 
each characteristic of sham acupuncture on the main effect 
estimate using the Stata command metareg. This command was also 
used to run a random-effects meta-regression to test the effect of 
routine versus protocoUed care on the main effect estimate for 
usual care control groups. The main effect estimate of each trial 
was determined using linear regression, and the coefficient and 
standard error for each trial were entered as the dependent 
variable in the random-effects meta-regression. 

A sensitivity analysis was performed excluding three trials by 
Vas et al. [12][13][14]. In our initial publication on effect size [2], 
we reported that these trials have very much larger effect sizes than 
average and that their exclusion resulted in heterogeneity 
becoming non-significant in the comparisons between acupunc- 
ture and sham. The trial of acupuncture for knee osteoarthritis by 
Berman et al. [,5] used a combined insertion and non-insertion 
method for sham acupuncture. As a sensitivity analysis, we 
performed the analysis with this trial reclassified as using non- 
penetrating needles on true acupuncture points as well. We also 
excluded trials where the risk of bias from unblinding was not 
classed as being low. As a final sensitivity analysis of non-sham 
controlled trials, we excluded the trials by Haake et al. [1 1] and 
Diener et al. [6] for which there was disagreement as whether the 
control arm constituted active control, which is not eligible for 
analysis [10]. AH analyses were conducted using Stata 12 (Stata 
Corp., College Station, TX). 



Results 

Sham acupuncture controls 

Trial-level characteristics for sham-controUed trials are de- 
scribed in Table 3. The majority of sham-controlled trials (80%) 
used needle-based sham acupuncture. The number of trials using 
penetrating or non-penetrating needles was similar: seven trials 
used non-penetrating needles and nine trials used penetrating 
needles. All trials using penetrating needles placed these outside 
true acupuncture points, while only one of seven trials using non- 
penetrating needles did so. 

Table 4 shows the effect sizes of sham-controlled acupuncture 
trials categorized by the type of sham. Acupuncture is significandy 
superior to sham irrespective of the type of sham control, both in 
the main analysis and in a sensitivity analysis excluding outlying 
studies. Table 4 also includes the results of the primary sensitivity 
analyses that excluded the Vas trials, which we had previously 
found to be outliers [2] .For example, not only was the effect size of 
the Vas trial for neck pain [13] about five times greater than the 
meta-analytic estimate, but between-trial heterogeneity was no 
longer statistically significant after excluding the Vas trials. Using 
the same rationale for exclusions, overall we found larger effect 
sizes were associated with acupuncture vs. non-penetrating sham 
needles (0.43; 95%CI: 0.01, 0.85) than vs. penetrating sham 
needles (0.17; 95%CI: 0.1 1, 0.23) although the difference between 
groups did not reach conventional levels of statistical significance. 

Statistical comparisons between types of sham are given in 
Table 5, which shows the results of the random-effects meta- 
regression for sham-controUed trials. While trials that used needles 
as sham did not differ significantly from trials with non-needle 
sham (P&0.2 for all comparisons), there is clear evidence of a 
greater effect size when acupuncture is compared against non- 
penetrating sham than when compared to penetrating sham. 
Trials using a penetrating needle had an effect size of —0.21 (95% 
C.I. —0.41, —0.01) standard deviations lower than trials that did 
not use a needle sham (p = 0.036). Trials that used penetrating 
needles for sham control had smaller effect sizes than those with 
non-penetrating sham or sham control without needles. The 
difference in effect size was -0.45 (95% C.I. -0.78, -0.12; 
p = 0.007). For the sensitivity analysis that excluded the Vas trials, 
this effect size reduced to -0.19 (95% C.I. -0.39, 0.01; 
p = 0.058). There were no significant differences between non- 
penetrating needles and sham techniques that did not involve 
needling. 

In further sensitivity analyses, reclassification of the Berman trial 
had Utde effect on our results. For example, the comparison of 
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Table 2. Types of non-sham control group by trial: categorised as "protocolled care" or "routine care". 





Trial 


Control Group 


Type of Control Group 


Foster (2007) [24] 


Advice and exercise: All three arms of the trial received advice and exercise. Patients received 
leaflet with information on knee osteoarthritis. Patients on NSAID therapy were allowed to 
continue with stable dose. Individualized exercises of progressive intensity for lower limb 
stretching, strengthening and balance (up to six 30-minute sessions over six weeks). Patients 
in the control arm did not receive verum or sham acupuncture. 


Protocolled 


Linde (2005) [3], Melchart 
(2005) [25] 


Waiting list control: Control patients were not permitted to have prophylactic treatment for 
12 weeks. All patients were allowed to treat acute headache as necessary {following 
current guidelines). 


Routine 


Thomas (2006) [33], 
baiter [Zuyjo) Lj4J, 
Vickers (2004) [23] 


General practitioner care: All patients received NHS treatment according to general 
practitioner's assessment and recommendation. Control patients did not receive 
acupuncture or any other specified interventions. 


Routine 


Berman (2004) [5] 


Education-attention control: Patients in this arm attended six two-hour group sessions 
based for arthritis self-management, and received periodic educational materials by mail. 
Patients in the acupuncture and sham acupuncture arms did not participate in this intervention. 


Routine 


Gherkin (2001) [35] 


Self-care education: Patients in this group received a book with information about back pain, 
treatment, improving quality of life and coping with emotional and interpersonal issues 
surrounding back pain. Patients also received two professionally-produced videos which 
addressed self-management of back pain and demonstrated exercises. Patients in the 
acupuncture and massage groups did not receive this educational material. 


Routine 


Scharf (2006) [26] 


Conservative therapy: Patients in the conservative therapy group had 10 visits with physicians 
and received prescriptions for either diclofenac {up to 150 mg/day) or rofecoxib {25 mg/day) 
up to week 23. Patients in this group who had "partially successful" results were offered the 
choice of attending an additional five visits. Patients in the verum acupuncture and sham 
acupuncture groups were permitted to take up to 150 mg/day of diclofenac for the first two 
weeks and a total of 1 g of diclofenac during the rest of the study. Patients in both acupuncture 
groups and in the conservative management group received up to six sessions of physiotherapy. 
All patients were prohibited from taking any analgesics other than diclofenac and rofecoxib and 
any corticosteroids. 


Protocolled 


uiener izuubj [o\ 


Standard migraine treatment: Control group patients were treated according to the guidelines 
of the German Migraine and Headache Society. Patients had six to seven visits in which standard 
treatment was established. First choice of treatment was beta blockers, followed by flunarizine, 
and then valproic acid. Acute medication use was permitted in all groups. 


Protocolled 


Haake (200/) [1 IJ 


Conventional therapy: Patients in the conventional therapy group were treated according to 
German guidelines. Conventional therapy patients had 10 visits with physician or physiotherapist 
where physiotherapy, exercise and/or similar treatments were offered. Patients in all three arms 
were permitted to take NSAIDs up to the maximum daily dose. 


Protocolled 


Williamson (2007) [36] 


Education and exercise: Patients in the control group were told they were in the "home exercise" 
group and received an exercise and advice leaflet. 


Routine 


vvlIT [ZVUDj lZo\, 

Brinkhaus (2006) [29] 


Waiting list control: Patients in the waiting list control group received no acupuncture treatment 
for eight weeks after randomization. All patients were allowed oral NSAIDs for pain as rescue 
medication. All patients were prohibited from taking corticosteroids or pain medication that 
acted on the central nervous system. 


Routine 


Witt (2006 - OA) [37], 
Witt (2006 - LBP) [38] 


Conventional treatment: Patients in the control group were not allowed to use any kind of 
acupuncture during the first three months. All patients were allowed to use additional 
conventional treatments as needed. 


Routine 


Jena (2008) [39], 

Witt (2006 - Neck Pain) [40] 


Conventional treatment: Patients in the control group were not allowed to use any kind of 
acupuncture during the first three months. All patients were allowed to use additional 
conventional treatments as needed. 


Routine 



doi:l 0.1 371/journal.pone.0093739.t002 



penetrating needle vs. non-needle or non-penetrating needle gave 
a difference in effect size of 0.18 rather than 0.19 and a p value of 
0.057 rather than 0.058. Excluding the four trials that were 
classified as "intermediate risk of unblinding" did not significantly 
change the effect sizes or p-values for most analyses. For example, 
the difference in effect size and 95% confidence interval when 
comparing penetrating needle sham to non-penetrating needle 
sham was —0.57 (—0.96, —0.18) in the main analysis, while the 
difference in effect size and 95% confidence for the same 
comparison was —0.57 (—0.98, —0.15) after excluding these four 
trials. However, the comparison with non-needle-based sham 
acupuncture included only one trial using non-neecUe sham. As a 
result, the standard error becomes much larger, and the p value 



non-significant. Excluding the trials at intermediate risk of 
unblinding and the outlying Vas trials gave very similar results 
to the analyses excluding the outlying Vas trials alone. For 
example, the central estimate for the comparison of penetrating 
needle with non-needle sham changes from —0.21 to —0.18. 
Statistical significance was lost, presumably because of the limited 
number of trials remaining in the analysis (see Table SI, 
Supporting Information). 

Non-sham controls 

Trial-level characteristics for trials without sham controls are 
described in Table 2. The majority of these trials (72%) were 
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Table 3. Trial-level Characteristics for Trials with Sham 
Acupuncture Control Groups, N = 20. 





Needle Used 


Yes 


16 (80%) 


No 


4 (20%) 


Penetrating Needle Used 


Yes 


9 (45%) 


No 


7 (35%) 


Non-needle 


4 (20%) 


True Acupuncture Points Used 


Yes 


8 (40%) 


No 


1 2 (60%) 


Superficial or Deep Sham 


Superficial 


8 (40%) 


Deep 


1 (5%) 


Non-penetrating sham 


11 (55%) 


Pain Type 


Low Bacl( Pain 


5 (25%) 


Migraine 


2 (10%) 


Neck 


3 (15%) 


Osteoarthritis 


5 (25%) 


Shoulder 


3 (15%) 


Tension-type Headache 


2 (10%) 



Frequency (%). 

doi:1 0.1 371 /journa!.pone.0093739.t003 



classified as routine care. Table 6 provides further details of the 
control groups, separately by pain type. 

The effect size for acupuncture in trials with routine care 
control (0.55, 95% CI 0.40, 0.70) was larger than when 
acupuncture was compared against protocoUed care (0.29, 95% 
CI 0.01, 0.58). Although the difference in effect size was large, it 
was not significant (difference in effect size = 0.26, 95% CI —0.05, 
0.57, p = 0.1). Removing the two studies [6] [1 1] in the sensitivity 
analysis had little effect on the effect size estimate (0.25, 95% CI 
—0.26, 0.76) for the comparison with protocoUed care. The 
difference in effect size between trials utilizing protocoUed vs. 
routine care was also simUar (0.29, 95% CI -0.13, 0.72). 



Discussion 

Principal findings 

Acupuncture was significandy superior to sham irrespective of 
the type of sham control and superior to non-sham control 
irrespective of whether that constituted routine or protocoUed 
care. That said, there were differences in effect sizes between trials 
with different control conditions. With regard to the types of sham 
control, we found that sham controls involving penetrating needles 
had smaller effect sizes than trials that did not use a needle control 
or where the needles in the control group did not penetrate the 
skin. An important implication is that the central estimates from 
our meta-analysis [2] may have underestimated the effects of 
acupuncture compared to sham. With regard to non-acupuncture 
controls, we found evidence that the effect size of acupuncture 
when compared to protocoUed care is smaller than when 
compared to the less intensive routine care, although differences 
did not reach statistical significance. 

There are two possible explanations for the differences in effect 
size by type of sham control: bias from unblinding and physiologic 
activity. It is plausible that penetrating needles are more credible 
to patients than non-penetrating approaches, such that patients 
are less likely to give biased responses on pain questionnaires. That 
said, there is no evidence in favor of such a hypothesis and 
considerable evidence against. In particular, the most common 
form of non-penetrating needle used was the "Streitberger" neecUe 
that has been carefuUy validated as a credible placebo in an 
empirical study. Indeed, study participants were unable to 
distinguish between the Streitberger needle and true acupuncture 
even when subject to both in crossover fashion [15]. The other 
explanation for our findings is that penetrating needles have 
important physiologic activity, that is, inserting an acupuncture 
needle superficially away from an acupuncture point may be less 
effective than deep insertion at a correct location, but nonetheless 
has some therapeutic activity against pain [16] [17]. 

Relationship to the literature 

There has been considerable interest in the literature regarding 
the appropriate choice of placebo controls for non-pharmacolog- 
ical therapies. One approach has been to investigate trials that 
included a placebo arm and a no-treatment arm, and then 
compare outcomes between these two, and in this way explore 
variations in the impact of the different types of placebo. An 
example of this is a Cochrane review of placebo controls covering 
a wide range of trials for different conditions, including some 
acupuncture trials [18]. In a sub-group analysis the authors found 
that trials using "physical placebos" (including sham acupuncture) 
were associated with greater placebo effects than trials with 



Table 4. Effect size of acupuncture compared to type of sham acupuncture control. 







Main Analysis 




Excluding Vas et al. trials [12] [13] [14] 




Number of Trials 


Effect Size 


Number of Trials 


Effect Size 


Needle sham 


16 


0.42 (0.19, 0.66) 


13 


0.22 (0.11, 0.33) 


Non-needle sham 


4 


0.38 (0.19, 0.57) 


4 


0.38 (0.19, 0.57) 


Non-penetrating needle 


7 


0.76 (0.31, 1.21) 


4 


0.43 (0.01, 0.85) 


Penetrating needle 


9 


0.17 (0.11, 0.23) 


9 


0.17 (0.11, 0.23) 


Non-needle and non-penetrating needle 


11 


0.63 (0.33, 0.94) 


8 


0.40 (0.18, 0.62) 



Estimates obtained using meta-regression. 
doi:l 0.1 371/journa!.pone.0093739.t004 
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Table 5. Difference in effect sizes between types of sham control. Estimates obtained using meta-regression. 







Main Analysis 






Excluding Vas et al. trials [12] [13] [14] 






No. of Trials* 


Change in Effect 
Size 


p value 


No. of Trials* 


Change in Effect 
Size 


p value 


Needle vs. Non-needle sham 


16 vs. 4 


0.02 (-0.49, 0.53) 


0.9 


13 vs. 4 


-0.17 (-0.43, 0.09) 


0.2 


Non-penetrating needle vs. Non-needle sham 


7 vs. 4 


0.35 (-0.28, 0.99) 


0.3 


4 vs. 4 


0.01 (-0.45, 0.47) 


1 


Penetrating needle vs. Non-penetrating needle 


9 vs. 7 


-0.57 (-0.96, -0.18) 


0.004 


9 vs. 4 


-0.19 (-0.47, 0.08) 


0.2 


Penetrating needle vs. Non-needle sham 


9 vs. 4 


-0.21 (-0.41, -0.01) 


0.036 


9 vs. 4 


-0.21 (-0.41, -0.01) 


0.036 


Penetrating needle vs. Non-needle or Non- 
penetrating needle 


9 vs. 1 1 


-0.45 (-0.78, -0.12) 


0.007 


9 vs. 8 


-0.19 (-0.39, 0.01) 


0.058 



*The number listed in the top row is the number of trials in the first comparison group. The number of trials listed in the bottom row Is the number of trials in the 
second comparison group. For example, there were 16 needle sham-controlled trials and 4 non-needle sham-controlled in the main analysis. 
doi:10.1371/journal.pone.0093739.t005 



pharmacological placebos [18]. This finding is consistent with the 
results of a trial that was specifically designed to compare a sham 
device (sham acupuncture) with an inert pUl, the sham device 
being associated with a greater reduction of self-reported pain 
[19]. These results provide supportive evidence for our finding 
that difierent types of sham control lead to different estimates of 
treatment effects. 

The data from the above Cochrane review of placebo controls 
were re-analysed by a different group of authors who observed that 
sham acupuncture interventions vs. no treatment have larger 
effects than other "physical placebos" vs. no treatment [20]. In a 
sub-analysis that is similar to what we report in this paper, they 
found that the standardised effect for acupuncture versus sham 
was similar for trials using penetrating sham needling (—0.43; 
95%CI: —0.59, 0.28) compared to trials using non-penetrating 
sham (-0.37; 95%CI: -0.70, 0.04) [20]. By contrast, we found 
significantly smaller effect sizes when acupuncture was compared 
to sham acupuncture with penetrating needles (0.17; 95%CI: 0.1 1, 
0.23) than when compared to non-penetrating needles (0.43; 
95%CI: 0.01, 0.85). The differences might be explained by 
differences in the trials included - our data involved only chronic 
pain trials of methodologically high quality - and the greater 
precision afforded by individual patient data meta-analysis: note 
that the wide confidence intervals in the Cochrane data are 
consistent with the main estimates from the current analysis. 



Table 6. Trial-level Characteristics for Trials with Non-sham 
Control Groups, N = 18. 



Study strengths and limitations 

Combining patient data from 29 high-quality trials in a single 
database provides us for the first time with sufficient power to 
explore the role of controls in trials of acupuncture for chronic 
pain, because the power of meta-regression is strongly influenced 
by the number of trials and their variation. We were unable to 
address questions as to the depth and location of sham needle 
placement as only one trial used deep sham needle insertion and 
all sham-controUed trials that used true acupuncture points 
avoided penetrating needles. While the difference in effect size 
between routine and protocoUed care is large and in the direction 
expected, it is associated with wide confidence intervals. Partially, 
this is due to the wide variety of non-sham controls, and the 
difficulty we had in categorizing them. 

Even with this large dataset we do not have a full understanding 
of the different physiologic and psychologic effects of sham 
acupuncture. One limitation within the field generally is that the 
mechanisms for a persistent effect of acupuncture on chronic pain 
are incompletely understood and therefore we have no clear idea 
of whether a sham control inadvertently activates these mecha- 
nisms or not. This lack of understanding about the physiological 
mechanisms of acupuncture limits any firm conclusions we can 
draw regarding the extent that any of the sham controls discussed 
above can be considered as a true 'placebo'. Moreover when 
implementing sham acupuncture trials, the outcome may also be 
influenced by factors not included in our analysis, such as the 
believabUity of the control, prior knowledge of patients about 
acupuncture, whether the true acupuncture group was treated 
identically, the extent that practitioners were able to maintain 
equipoise, and practical implementation issues, such as how 
carefully the ring that comes with the Streitberger needle [15] was 
taped in place. 



Pain Type Routine Care ProtocoUed Care Total 



Headache 2 0 2 

Migraine 1 1 2 

Tension-Type Headache 1 0 1 

Osteoarthritis 4 2 6 

Lower Back Pain 3 2 5 

Neck Pain 2 0 2 

Total 13(72%) 5(28%) 18(100%) 

Frequency (%). 

doi:l 0.1 371/journal.pone.0093739.t006 



Implications for research 

The research question remains the primary determinant on 
choice of control. In a strategy document developed with a range 
of collaborators using consensus methods, a useful distinction has 
been drawn between efficacy trials that seek to determine whether 
there are specific effects beyond the placebo in an ideal treatment 
environment, and effectiveness trials that seek to determine the 
overall impact of acupuncture in which specific and non-specific 
effects are combined [21]. Moreover research questions investi- 
gating the value of specific point location need to have sham 
needles located away from true acupuncture points while research 
questions testing skin penetration require non-penetrating sham 
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needle controls applied at the same points as in the true 
acupuncture group. 

The choice of a sham acupuncture control needs to be informed 
by consideration of the likely impact of the sham intervention. In 
the past, judgments on this have often used expert opinion on 
putative physiological activity of a sham control, even though we 
have yet to understand the mechanism(s) of the action of 
acupuncture [17]. A number of commentators have speculated 
that penetrating sham n(xdling may be physiologically active and 
thus be an inappropriate sham control [16]. Our results provide 
support for this contention, suggesting that needle penetration 
should be avoided as a sham technique to control for non-specific 
effects associated with acupuncture in trials involving chronic pain 
patients. However sham acupuncture involving penetrating 
needles may well have a place when addressing questions of point 
specificity in explanatory trials. We are more cautious with regard 
to recommending the use of non-penetrating needles. Many forms 
of Japanese acupuncture use shallow insertion or non-insertion 
(the toya hari method) [22]. Using non-penetrating needles in 
controlled trials is not without its challenges: although apparendy 
less active than other types of sham, we cannot assume that non- 
penetrating needles have complete physiologic inactivity; further- 
more, there are practical questions regarding whether to enroll 
only acupuncture-naive patients and whether practitioners can 
maintain equipoise in large trials over reasonable periods of time. 

When sham acupuncture is not used, the choice of control is 
clearly driven by the research question. For instance, in the UK 
National Health Scr\'ice (NHS) trial of acupuncture for chronic 
headache, the study question of Vickers et al was related to the 
effects of making acupuncture more widely available in primary 
care, a pragmatic comparison of "use acupuncture" and "avoid 
acupuncture" [23]. On the other hand, Foster et al. were 
interested in the impact of acupuncture when added to an existing 
rehabilitation program [24]. Yet our findings have clear implica- 
tions for sample size calculations, with larger sample sizes needed 
in trials where care in the control arm is carefully specified. 

Conclusion 

From a large database of individual patient data from high- 
quality randomized trials, we found acupuncture to be significandy 
superior to control irrespective of the subtype* of control. AVhen 
compared against sham, trials with penetrating needles reported 
lower effect sizes for acupuncture than trials with non-penetrating 
needles or those that used non-needle sham. This suggests that 
penetrating needles have important physiologic activity, even 
when inserted superficially away from true acupuncture points. 
Accordingly, we recommend that this type of sham be avoided. In 
trials without sham control, we found that the effect size likely 
depends on the intensity of treatment in the control group, with 
smaller differences between acupuncture and protocol guided 
programs of treatment than between acupuncture and routine 
care. While the choice of control should be driven by the study 
question, these findings can help inform study design in 
acupuncture, particularly with respect to sample size. 

Supporting information 

Table SI Sensitivity analyses. 
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